Skip to content

New braille IPC APIs: BrailleMirror, DirectBrailleWindow, and named-pipe server#19856

Draft
trypsynth wants to merge 6 commits intonvaccess:masterfrom
trypsynth:new-braille-apis
Draft

New braille IPC APIs: BrailleMirror, DirectBrailleWindow, and named-pipe server#19856
trypsynth wants to merge 6 commits intonvaccess:masterfrom
trypsynth:new-braille-apis

Conversation

@trypsynth
Copy link
Copy Markdown
Contributor

@trypsynth trypsynth commented Mar 25, 2026

Link to issue number:

Closes #19843

Summary of the issue:

External processes (e.g. RIM's elevated service component) have no supported way to receive a live mirror of NVDA's braille output or to act as a direct braille display for a foreground window. The existing pre_writeCells / filter_displayDimensions / decide_enabled extension points are not sufficient on their own: they require in-process code, have no HWND-scoping, and block the NVDA main thread if pipe I/O is done naively.

Description of user facing changes:

None for end users. NVDA Remote users with braille displays can continue using braille normally; the refactor is transparent.

Description of developer facing changes:

Two new public APIs in braille:

  • BrailleMirror - abstract class. Register with braille.registerMirror(); receives every display() call and optionally caps the display width via numCells(). Unregister with braille.unregisterMirror().
  • DirectBrailleWindow - HWND-scoped class. While its window is foreground, NVDA rendering is suspended (decide_enabled) and braille gestures are intercepted (decide_executeGesture) and forwarded to onGesture(). Subclass and call activate()/deactivate().
  • braille.injectGesture(gesture) - helper to execute a BrailleDisplayGesture via inputCore.manager, swallowing NoInputGestureAction.

New module braillePipeServer exposes both APIs over a named pipe (\.\pipe\nvda_braille / \.\pipe\nvda_braille_secure). Wire protocol: 4-byte LE uint32 length-prefix + UTF-8 JSON. Pipe DACL grants NT AUTHORITY\SYSTEM read/write so RIM's elevated service can connect on the normal desktop.

Description of development approach:

  • registerMirror lazily hooks pre_writeCells and filter_displayDimensions (shared single handler); unhooks when last mirror unregisters.
  • DirectBrailleWindow hooks decide_enabled and inputCore.decide_executeGesture per instance; optionally hooks filter_displayDimensions if numCells > 0.
  • braillePipeServer uses one accept-loop thread + one _AsyncWriter background thread per connection so pipe I/O never touches the NVDA main thread.
  • _remoteClient refactored: FollowerSession now uses _FollowerBrailleMirror instead of direct pre_writeCells wiring; localMachine.setBrailleDisplaySize / _handleFilterDisplayDimensions removed; pre-existing filter_displayDimensions leak on transport close fixed.

Testing strategy:

30 unit tests added in tests/unit/test_braille/test_mirrorAndDirectWindow.py covering:

  • BrailleMirror register/unregister lifecycle and extension-point hook/unhook
  • Display forwarding to one and multiple mirrors
  • numCells capping logic (zero, smaller, larger, smallest-wins)
  • injectGesture happy path and NoInputGestureAction swallow
  • DirectBrailleWindow activate/deactivate idempotency, decide_enabled and decide_executeGesture routing, filter_displayDimensions registration and capping

Manual testing needed: NVDA Remote with a physical braille display on both sides. I've locally tested with a 20 cell on both sides individually, but am unable to test both at the same time

Known issues with pull request:

RIM client-side integration is not yet complete. Do not merge until the RIM release is coordinated.
LeaderSession in NVDA Remote still wires decide_executeGesture directly rather than via DirectBrailleWindow; should this be migrated in core as well? Needs discussion.

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposed NVDA Core APIs: Braille Mirror and Direct Braille

4 participants